Pacific Northwest National Laboratory, Richland, WA Exploration of complex high-dimensional spaces presents significant challenges in fields such as molecular discovery, process optimization, and supply chain management. Genetic Algorithms (GAs), while offering significant power for creating new candidates' spaces, often entail high computational demands due to the need for evaluation of each new proposed solution. On the other hand, Deep Kernel Learning (DKL) efficiently navigates the spaces of preselected candidate structures but lacks generative capabilities. This study introduces an approach that amalgamates the generative power of GAs to create new candidates with the efficiency of DKL-based surrogate models to rapidly ascertain the behavior of new candidate spaces. This DKL-GA framework can be further used to build Bayesian Optimization (BO) workflows. We demonstrate the effectiveness of this approach through the optimization of the FerroSIM model, showcasing its broad applicability to diverse challenges, including molecular discovery and battery charging optimization. Manufacturing and chemical engineering often require complex resource allocation and control problems. Optimization in molecular spaces is crucial because it enables the discovery of new compounds with desired properties, leading to breakthroughs in various fields such as pharmaceuticals, materials science, and energy storage. Efficiently exploring molecular spaces allows researchers to identify novel molecules that can serve as effective drugs, advanced materials with superior properties, or catalysts for chemical reactions. By optimizing molecular spaces, we can uncover hidden relationships and patterns that lead to more efficient and targeted experimentation, reducing costs and time associated with traditional trial-and-error methods.

Large Language Models (LLMs) have been applied to many research problems across various domains. One of the applications of LLMs is providing question-answering systems that cater to users from different fields. The effectiveness of LLM-based question-answering systems has already been established at an acceptable level for users posing questions in popular and public domains such as trivia and literature. However, it has not often been established in niche domains that traditionally require specialized expertise. To this end, we construct the NEPAQuAD1.0 benchmark to evaluate the performance of three frontier LLMs -- Claude Sonnet, Gemini, and GPT-4 -- when answering questions originating from Environmental Impact Statements prepared by U.S. federal government agencies in accordance with the National Environmental Environmental Act (NEPA). We specifically measure the ability of LLMs to understand the nuances of legal, technical, and compliance-related information present in NEPA documents in different contextual scenarios. For example, we test the LLMs' internal prior NEPA knowledge by providing questions without any context, as well as assess how LLMs synthesize the contextual information present in long NEPA documents to facilitate the question/answering task. We compare the performance of the long context LLMs and RAG powered models in handling different types of questions (e.g., problem-solving, divergent). Our results suggest that RAG powered models significantly outperform the long context models in the answer accuracy regardless of the choice of the frontier LLM. Our further analysis reveals that many models perform better answering closed questions than divergent and problem-solving questions.

We propose a physics-informed machine learning method for uncertainty quantification in high-dimensional inverse problems. In this method, the states and parameters of partial differential equations (PDEs) are approximated with truncated conditional Karhunen-Lo\`eve expansions (CKLEs), which, by construction, match the measurements of the respective variables. The maximum a posteriori (MAP) solution of the inverse problem is formulated as a minimization problem over CKLE coefficients where the loss function is the sum of the norm of PDE residuals and the $\ell_2$ regularization term. This MAP formulation is known as the physics-informed CKLE (PICKLE) method. Uncertainty in the inverse solution is quantified in terms of the posterior distribution of CKLE coefficients, and we sample the posterior by solving a randomized PICKLE minimization problem, formulated by adding zero-mean Gaussian perturbations in the PICKLE loss function. We call the proposed approach the randomized PICKLE (rPICKLE) method. For linear and low-dimensional nonlinear problems (15 CKLE parameters), we show analytically and through comparison with Hamiltonian Monte Carlo (HMC) that the rPICKLE posterior converges to the true posterior given by the Bayes rule. For high-dimensional non-linear problems with 2000 CKLE parameters, we numerically demonstrate that rPICKLE posteriors are highly informative--they provide mean estimates with an accuracy comparable to the estimates given by the MAP solution and the confidence interval that mostly covers the reference solution. We are not able to obtain the HMC posterior to validate rPICKLE's convergence to the true posterior due to the HMC's prohibitive computational cost for the considered high-dimensional problems. Our results demonstrate the advantages of rPICKLE over HMC for approximately sampling high-dimensional posterior distributions subject to physics constraints.

RICHLAND, Wash.-Hurricane Ida was among the most intense and damaging hurricanes in Louisiana's history. The violent storm rose to a Category 1 hurricane on Friday, August 27. It then climbed another two categories in two days, jumping from Category 3 to 4 in only an hour. Thankfully, forecasting models help us predict when, where, and how strongly hurricanes may strike. Accurately predicting the brief windows in which these violent storms surge and strengthen is a lingering blind spot within the hurricane forecasting community.

If a person in the developing world severely fractures a limb, they face an impossible choice. An improperly healed fracture could mean a lifetime of pain, but lengthy healing time in traction or a bulky cast results in immediate financial hardship. That's why Pacific Northwest National Laboratory (PNNL) machine learning scientists leaped into action when they learned they could help a local charity enable patients in the developing world to walk within one week of surgery--even when fractures are severe. For more than 20 years, the Richland, Washington-based charity SIGN Fracture Care has pioneered orthopedic care, including training and innovatively designed implants that speed healing without real-time operating room X-ray machines. During those 20 years, they've built a database of 500,000 procedure images and outcomes that serves as a learning hub for doctors around the world.

Do you want to create a legacy of meaningful research for the greater good? Do you want to lead and contribute to work in support of an organization that addresses some of today's most challenging problems that face our Nation? Then join us in the Data Sciences and Analytics Group at the Pacific Northwest National Laboratory (PNNL)! For more than 50 years, PNNL has advanced the frontiers of science and engineering in the service of our nation and the world in the areas of energy, the environment and national security. PNNL is committed to advancing the state-of-the-art in artificial intelligence through applied machine learning and deep learning to support scientific discovery and our sponsors' missions.

Pacific Northwest National Laboratory is joining forces with two other research powerhouses to pioneer a new $5.5 million research center created by the U.S. Department of Energy to focus on the biggest challenges in artificial intelligence. The Center for Artificial Intelligence-Focused Architectures and Algorithms, or ARIAA, will promote collaborative projects for scientists at PNNL in Richland, Wash., at Sandia National Laboratories in New Mexico, and at Georgia Tech. PNNL and Sandia are part of the Energy Department's network of research labs. ARIAA will be headed by Roberto Gioiosa, a senior research scientist at PNNL. As center director, he'll be in charge of ARIAA's overall vision, strategy and research direction.

Locally adaptive activation functions with slope recovery term for deep and physics-informed neural networks Ameya D. Jagtap 1, Kenji Kawaguchi 2 and George Em Karniadakis 1,3, 1 Division of Applied Mathematics, Brown University, 182 George Street, Providence, RI 02912, USA. 2 Massachusetts Institute of T echnology, 77 Massachusetts Ave, Cambridge, MA 02139, USA. 3 Pacific Northwest National Laboratory, Richland, WA 99354, USA.Abstract We propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep and physics-informed neural networks. The local adaptation of activation function is achieved by introducing scalable hyper-parameters in each layer (layer-wise) and for every neuron separately (neuron-wise), and then optimizing it using the stochastic gradient descent algorithm. Introduction of neuron-wise activation function acts like a vector activation function as opposed to the traditional scalar activation function given by fixed, global and layer-wise activations. In order to further increase the training speed, an activation slope based slope recovery term is added in the loss function, which further accelerate convergence, thereby reducing the training cost. For numerical experiments, a nonlinear discontinuous function is approximated using a deep neural network with layer-wise and neuron-wise locally adaptive activation functions with and without the slope recovery term and compared with its global counterpart. Moreover, solution of the nonlinear Burgers equation, which exhibits steep gradients, is also obtained using the proposed methods. On the theoretical side, we prove that in the proposed method the gradient descent algorithms are not attracted to sub-optimal critical points or local minima under practical conditions on the initialization and learning rate. Furthermore, the proposed adaptive activation functions with the slope recovery are shown to accelerate the training process in standard deep learning benchmarks using CIFAR-10, CIFAR-100, SVHN, MNIST, KMNIST, Fashion-MNIST, and Semeion data sets with and without data augmentation. Keywords: Machine learning, bad minima, stochastic gradients, accelerated training, PINN, deep learning benchmarks. 1. Introduction In recent years, research on neural networks (NNs) has intensified around the world due to their successful applications in many diverse fields such as speech recognition [13], computer vision [16], natural language translation [25], etc. Training of NN is performed on data sets before using it in the actual applications.

COLUMBUS, OHIO – Ohio crews cleaning up a massive former Cold War-era uranium enrichment plant in Ohio plan this summer to deploy a high-tech helper: an autonomous, radiation-measuring robot that will roll through kilometers of large overhead pipes to spot potentially hazardous residual uranium. Officials say it's safer, more accurate and tremendously faster than having workers take external measurements to identify which pipes need to be removed and decontaminated at the Portsmouth Gaseous Diffusion Plant in Piketon. They say it could save taxpayers tens of millions of dollars on cleanups of that site and one near Paducah, Kentucky, which for decades enriched uranium for nuclear reactors and weapons. The RadPiper robot was developed at Carnegie Mellon University in Pittsburgh for the U.S. Department of Energy, which envisions using similar technology at other nuclear complexes such as the Savannah River Site in Aiken, South Carolina, and the Hanford Site in Richland, Washington. Roboticist William "Red" Whittaker, who began his career developing robots to help clean up the Three Mile Island nuclear power accident and now directs Carnegie Mellon's Field Robotics Center, said technology like RadPiper could transform key tasks in cleaning up the country's nuclear legacy.

Teen scientists use machine learning and neural networks to detect and diagnose diseases, track space debris, design drones and justify conclusions at Intel ISEF 2017. While sentient computer beings like HAL from the classic 2001: A Space Odyssey or Samantha from the 2013 film Her may still be on the distant horizon, some forms of artificial intelligence (AI) are already improving lives. At the 2017 Intel International Science and Engineering Fair (ISEF) – where nearly 1,800 high school students gathered to present original research and compete for more than $4 million in prizes – the next generation of scientists used machine learning and artificial neural networks to find solutions to some of today's most vexing problems. "AI is critical to our future," said Christopher Kang, a budding computer scientist from Richland, Washington, who won an ISEF award in the robotics and intelligent machines category. "Humans have a limit as to how much data we can analyze," he said.